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EXECUTIVE SUMMARY 



Since 1972 , the Office of Vocational and Adult Education 
(OVAE) of the U.S. Department of Education (ED) has provided 
support for the operation of the National Network for Curriculum 
Coordination in Vocational and Technical Education (the Network), 
a discretionary program consisting of six regional Curriculum 
Coordination Centers and a network of State Liaison Representa- 
tives (SLRs), whose time is donated by state departments of voca- 
tional education to participate in Network activities. The broad 
purpose of the program is to provide a mechanism for sharing 
information on vocational curriculum materials in order to avoid 
unwarranted duplication of effort in curriculum development acti- 
vities, increase the transportability of curriculum materials, 
and enhance the quality of state and local curriculum development 
efforts. 



Purposes of the Study 

This report is the second phase of an analysis of the opera- 
tions, management, and impacts of the Network in addressing its 
purpose. Specific issues addressed in this report include: (1) 
the types and extent of impacts the Centers have had on their 
users; (2) the nature and effects of relationships developed 
between the Network and other organizations involved in voca- 
tional education; and (3) factors that have affected program 
impacts. These issues were investigated through a variety ot 
data collection activities, including case studies of the six 
Curriculum Coordination Centers, participation in regional 
meetings of Center staff and SLRs, visits to other organizations 
involved in vocational education, and telephone interviews with 
selected SLRs and other state and local vocational educators. 



Study Conclusions and Recommendations 

Major findings that emerged from the study's data collection 
and analysis activities include the following: 

• The major purpose of the Network — to create a 
structure through which curriculum materials and 
resources would be readily available to practi- 
tioners—has been accomplished. 

# The ready availability of materials through the 
Network has resulted in savings of time and money to 
the states in their curriculum- related activities 
and has reduced duplication of effort in curriculum- 
development efforts. 
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• A chief benefit of the program has been the creation 
of a nationwide communications network that has 
facilitated the spread of curriculum materials and 
resources, as well as new approaches and technolo- 
gies, to vocational educators at the state and local 
levels. 

• The federal investment in the program has leveraged 
a considerable amount of state support for the 
Centers and Network , including funding for indiv- 
idual Centers and donation of SLR time for conduct 
of Network activities. 

The major recommendations concerning the program include the 
following : 

• The Network should consider implementing a curri- 
culum review procedure that would provide endorse- 
ment of materials meeting high standards of quality. 

• Increased outreach efforts would be useful in 
enhancing the program* s capacity to accomplish its 
curriculum coordination objectives 

• The Network should consider implementing a reporting 
system that would provide a clearer picture ot 
Center and Network accomplishments. 

• Individual Centers should be encouraged to adopt 
particular areas of emerging technologies as a 
special concentration in addition to their base 
library resources, a strategy that would provide 
users with a specified place tor obtaining needed 
materials in such high-cost areas as robotics or 
computer-assisted machining. 
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I. INTRODUCTION 



Since 1972, the Office of Vocational and Adult Education 
(OVAE) of the U.S. Department of Education (ED) has provided 
support for the operation of the National Network for Curriculum 
Coordination in Vocational and Technical Education (the Network), 
a discretionary program consisting of six regional Curriculum 
Coordination Centers (CCCs) and a network of State Liaison Repre- 
sentatives (SLRs), whose time is donated by state departments of 
vocational education to participate in Network activities. The 
broad purposes of the Network, as articulated by the program's 
founders, ar«; 

• To provide a mechanism for the sharing of informa- 
tion on curriculum materials available and under 
development 

• To develop and recommend guidelines for curricula 
and curriculum development with the ultimate goal of 
increasing the effectiveness of curriculum materials 
and enhancing their transportability 

• To establish and maintain a system for determining 
curriculum needs in vocational- technical education 
and reporting conclusions to the field 

• To coordinate activities ir. curriculum development, 
dissemination, and utilization with the aim of 
avoiding unwarranted duplication, enhancing quality 
of effort, increasing the transportability of curri- 
culum materials, and improving the acceptance and 
use of curriculum materials . 

• To report these curriculum coordination efforts to 
the field 

These purposes have been translated into operational objectives 
to which each Center responds in the competitive procurement 
process under which Center contracts are awarded. These opera- 
tional objectives for the six Centers (with minor variations 
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across the three Requests for Proposals (RFPs) under which the 



Centers operate) are as follows: 

• To increase the availability of curriculum 
information and materials to instructional program 
improvement personnel 

• To promote the adoption and adaptation of curriculum 
materials developed with Federal, state, and local 
funds in order to minimize duplication 

• To improve the quality of vocational educational 
curriculum and dissemination services provided to 
public, private, and proprietary schools 

• To establish linkages with other organizations or 
information-sharing systems in the vocational 
education community (RFP 84-007) 



Purposes of the Study 

The study whose findings are presented in this report is the 
second phase of an analysis of the operations, management, and 
impacts of the Network in addressing these objectives. An 
earlier report 1 presented the findings of a descriptive study of 
the operations and management of the Network, .including informa- 
tion on (1) the structure of the Network, (2) the major catego- 
ries of services and activities provided by the regional Centers 
to state and local vocational educators, (3) the users of Network 
services, and (4) factors that influence the effectiveness of the 
Network in achieving its purposes. 
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1 Becky Jon Hayward, Anne H. Hastings, and Linda A. LeBlanc, 
Operations and Management of the National Network for Curriculum 
Coordination in Vocational and Technical Education (Reston, VA: 
Advanced Technology, Inc., October .1984). 
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This report focuses on the impacts of the regional Centers 
on vocational educators and the Network's overall contribution to 
curriculum-related activities and initiatives in vocational 
education. Specific issues addressed in this report include: 

• The types and extent of impacts the Centers have had 
on various categories of users: SLRs, other state 
vocational educators , local vocational education 
administrators and teachers , and vocational educa- 
tion teacher educators 

• The nature and effects of relationships developed 
between the Network and other organizations involved 
in vocational education 

o The factors that- have affected program impacts 
It is important to note, in reviewing the findings presented 
in this report, that while the underlying rationale for the Net- 
work's existence has to do with improvement in the quality and 
outcomes of vocational education generally, the objectives under 
which the Centers operate are quite specific and process ori- 
ented. The focus of tnis impact study has been governed by the 
objectives established by OVAE for the regional Centers. Thus 
the Network's effectiveness is assessed here by determining the 
following : 

• Whether the federal investment in the program has 
resulted in wider availability of curriculum and 
instructional materials across the country 

• The extent to which states share information and 
resources and thereby reduce the costs and time 
required for curriculum development or adoption 

» The efficiency and effectiveness of dissemination of 
curriculum and instructional materials within and 
among states 

• The extent to which the Centers and Network have 
established viable relationships with other voca- 
tional education organizations 



While some broader effects and implications are addressed in the 
final chapter of this report, the report primarily focuses on 
evidence of the extent to which the program's specific objectives 
have been realized. 

In order to provide a context for the findings reported in 
later chapters, in the next section of this chapter we provide a 
summary description of the program based upon information 
retained in both phases of the analysis. 

Summary of the Program 

The Network was created in 1972 to provide a mechanism for 
state and local education agencies to coordinate curriculum 
activities and share curriculum resources. The Network is 
structured around six regional Curriculum Coordination Centers 
intended to serve the states and territories within their 
regions* (Appendix A lists the regions and states served by each 
Center.) Each state or territory is formally linked to one ot 
the six Centers through an SLR, who is appointed by the State 
Director of Vocational Education. The SLR serves as the primary 
point of contact for persons in the state who need Network 
services, representing the state to the Center and the Center to 
the state. Coordination among the Centers is provided by the 
seven-member Directors 1 Council , comprising the six Center 
Directors and one ex-of f icio member representing ED. While the 
six regional Centers, the 57 SLRs, and the Directors 1 Council 
constitute the formal structure of the Network, actual Network 
activities and outcomes have also been influenced by formal and 
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ad hoc relationships the Centers have developed over time with 
other organizations or information-sharing systems, such as the 
National Center for Research in Vocational Education (NCRVE) and 
a number of vocational education consortia. 

Program funding and admin is t rat ion . The Network is funded 
under the provisions of the Programs of National Significance 
(Subpart 2, Part B of the Vocational Education Amendments of 
1962, as amended), which authorizes federal discretionary funds 
for extending and improving vocational education. While the 
amount of annual federal support of the Network has varied over 
its 13-year history, in recent years federal funding has remained 
relatively stable at approximately $775,000. In 1984 the federal 
investment in the Network was $773,494, with individual Center 
budgets ranging from $88,762 to $174,804. 

Each Center operates under a three-year contract adminis- 
tered by the Program Improvement Systems Branch of OVAE. The 
contract for the firs ; - year's performance period is competitively 
awarded, with two subsequent years covered by noncompetitive 
continuations. Center contract periods are staggered such that 
each year two of the six Centers are recompeted. This phased 
contracting procedure has two effects. First, variability in the 
amount of discretionary funds available to OVAE at the time of 
each competition causes some of the variation in the size of the 
Centers 1 budgets. In years when federal discretionary funds are 
scarce, RFPs reflect a somewhat reduced scope of work , often 
eliminating one or more regional meetings, reducing or eliminat- 
ing technical assistance visits, or deleting funds for a Regional 



15 



6- 



Communique . A second result of the phased contracting is that 
some part of the Network can respond each year to new develop- 
ments and evolving federal priorities. 

The Network's Directors 1 Council has several functions. One 
is to provide recommendations, to ED concerning national priori- 
ties and policies affecting the Centers. Another is to facili- 
tate linkages and coordinated efforts between the Network and 
other organizations involved in vocational education program 
improvement. The Council convenes three times each year to 
review new developments and advise ED on whether new activities 
should be authorized. However, the degree to which the 
Directors' Council can coordinate the work of the Network is 
constrained. One reason is that, while agreeing on the 
overriding goals of the Network, Council members represent their 
own Centers and thus bring differences in philosophy and 
priorities to the Council. They are also obligated to fulfill 
the terms of individual Center contracts, which vary in the 
specific activities to be pursued and which do not explicitly 
authorize joint determination of Network priorities. 

With its federal funds each Center is contractually bound to 
perform five major tasks: 

• Participate as a member of the Directors' Council 

• Serve as a facilitator to the consortium states in 
the region 

• Abstract the Center' s library materials for the 
national computerized curriculum data base and 
forward updates to NCRVE 

• Prepare and submit periodic progress reports to ED 

• Provide annual summary impact information to ED 
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Under the competitive procurement process, each Center proposes 
its own approach for serving as a facilitator to states in its 
region. The variations in the types of activities each Center 
undertakes and the manner in which they are carried out can be 
attributed to differing perceptions by Center Directors regarding 
the proper role of the Centers, the specialized needs of the 
states they serve, and differences in the amount of federal 
funding each Center receives. For example , some Directors 
believe a Center should serve states essentially as a curriculum 
library. Others focus on a larger array of curriculum-related 
initiatives, (e.g., vocational education curriculum evaluation, 
new approaches to curriculum development). This "expansionist" 
view of the Center's role may be attributable not only to the 
level of OVAE funding or a Director's perception of the Center's 
appropriate role but also to the financial support some Centers 
receive from the state departments of vocational education in 
which those Centers are located. 

Center activities . While some of the activities ot the 
Centers address multiple purposes, the activities can be grouped 
under three broad classifications: information resources, 
capacity building, and linkage and outreach. An overview of 
these classes of activities follows.. 

In the area of information resources the Centers provide a 
number of services. The core function of each Center is to serve 
as a library of curriculum materials. SLRs send curriculum 
materials that have been developed in their states to their 
regional Centers, which categorize and distribute the materials 



to other states upon request. In past years, states have been 
asked to send enough copies of materials to their Center to allow 
dissemination to other Centers and all 57 SLRs. This practice 
has permitted virtually every state to accumulate its own library 
of curriculum materials developed in other states. The growth of 
such curriculum resource centers or libraries has enabled 
numerous states to satisfy many of their curriculum needs using 
only their own resources, contacting the Center only when state- 
held materials proved inadequate. 

In recent years some states have found the request to supply 
the Network with sufficient copies for all Centers and all SLRs 
to be financially or logistically too burdensome. In such 
instances sufficient copies may be provided only for the six 
Centers or even only for the state's regional Center. These 
Center-held materials are typically loaned to requesting states 
for a short period of time for their preview or use as a 
resource. Further, some states that sell state-developed 
curriculum materials do not share materials with the Network, 
although this arrangement appears to be changing, with more 
selling states sharing their materials tor preview. Addition- 
ally, Centers purchase copies of some materials (e.g., consortia 
catalogs) for preview use by their states. 

The Centers do not typically screen incoming materials for 
quality. Most Center staff members believe that any such 
centralized screening might eliminate from the resource base 
certain materials that could have some utility in a local setting 
whose vocational education needs and circumstances were unknown 
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to the Center. Some Centers are making changes in this regard , 
however. For example , one Center is asking SLRs to rate the 
quality of the materials they review so that others requesting 
materials will have some information on how the materials are 
viewed. 

To facilitate distribution of materials , most Centers 
prepare catalogs or bibliographies that states can use to 
identify potentially useful materials and request loans ot those 
materials for previewing. Some Centers use microfiche in this 
cataloging and previewing function, although others have found 
microfiche unacceptable. Computerization of catalogs is an 
emerging trend within the Network. At least two Centers are in 
the process of making their catalogs, task listings, and other 
materials available on-line to enable client states to examine 
materials directly through an electronic bulletin board, without 
waiting for loan copies, and another Center routinely sends 
catalogs on disks to persons requesting them. Some Centers, in 
fact, provide incentives to their state.s to participate in the 
electronic systems by offering start-up funding for terminals and 
staff training and holding conferences to acquaint state staff 
with system capabilities. 

This automation initiative is also an important factor in 
another central information-resource function of the Centers, 
performing regional or national searches for materials for state 
and local requestors. While most Centers still perform telephone 
and paper searches, particularly for materials still under devel- 
opment, they now rely to a significant extent on the Vocational 
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Education Curriculum Materials (VECM) database, an automated 
curriculum information system developed as a joint Network/NCRVE 
initiative. Each SLR is encouraged to abstract newly developed 
materials and forward the abstracts- to the regional Center, where 
they are edited and subsequently transmitted to NCRVE, for 
inclusion in this database. SLRs are also being encouraged to 
access VECM directly without going through the regional Center. 

The Centers also use electronic mail, primarily through 
the Adult and Vocational Education Electronic Mail Network 
(ADVOCNET) . Managed by NCRVE, ADVOCNET allows state and local 
vocational educators to transmit messages, documents, meeting 
announcements, and other information throughout the country. 

Capa c i ty- bu i Id i ng activities, targeted primarily toward the 
SLRs, and, by extention, to state and local vocational educators, 
include the annual Concurrent Meeting of all 57 SLRs as well as 
one or more Regional Meetings each year. The Concurrent Meeting 
brings all SLRs together for a week that includes seminars on new 
approaches to curriculum development and dissemination, visits to 
exemplary vocational education institutions, and sessions on 
strategies for increasing the effectiveness of SLRs within their 
state systems. Training sessions to introduce new SLRs to the 
Network's objectives and activities are held one day prior to the 
annual Concurrent Meeting. 

Capacity building is also one focus of the Regional Meetings 
of SLRs. At these meetings, which sometimes are co-sponsored by 
two Centers, SLRs are apprised of new curriculum-related develop- 
ments through seminars, presentations, and workshops. The intent 
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is that SLRs will return from a Regional Meeting with new know- 
ledge that is then applied, or at least further pursued, within 
their own states. 

: *~ In addition to these SLR-focused meetings, the Centers 

perform other functions intended to build state and local users 1 
capacity. Technical assistance to state and local vocational 
educators is provided by the Centers through inservice training, 

♦ - 

workshops, and consultation. Due to funding constraints, 
however, the number of technical assistance visits reported by 
the Centers has steadily declined over the last three years, from 
49 in 1982 to 32 in 1984. Another significant capacity-building 
activity of the Centers has been to assist individual states in 
the establishment of state curriculum laboratories and resource 
centers. 

Centers inform potential clients of their informat ion 
services and capacity building through a number of outreach and 
linkage efforts. Some Centers receive federal support for 
production of a Regional Communique , mailed to state and local 
vocational educators, that includes descriptions of Center 
services and articles on current issues in vocational education. 

f Some of the Centers have secured state funding to produce 

t 
t 

L_ descriptive brochures about their services for distribution to 

• potential Center users. In many states SLRs make presentations 
about Center services at workshops and other meetings. 

f 

i Another means of Network outreach is represented by the SLRs 

L 

through their ongoing communications with vocational educators in 
their states. Distributing materials/ responding to local 

I 

! 
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i requests, providing technical assistance, and other Center- 

supported, SLR-executed activities constitute important outreach 
mechanisms. Displays announcing Network services are often used 
by SLRs and Center staff at state and local conferences and 
workshops to inform vocational educators about the Network. 

Over the last decade, the Network has also established link- 
ages with such organizations as NCRVE, the Mid-America Vocational 
Curriculum Consortium (MAVCC), the Vocational-Technical Education 
Consortium of States (V-TECS), and other organizations involved 
in vocational education. The relationship ot NCRVE to the 
Network cuts across all six Centers and is formally established 
through the VECM database, the ADVOCNET system, and other joint 
projects. The Network's relationships with NCRVE, curriculum 
consortia, and other related organizations help spread informa- 
tion about the Network to a wider range of potential users and 
facilitate the sharing of products and other resources. 

Role of the SLRs . SLRs are in many ways the key to the 
entire Network. The time SLRs devote to the Network is donated 
by their states, with Center contracts reimbursing only their 

u travel expenses for attending Regional and Concurrent meetings. 

I Many SLRs occupy key positions in the state vocational education 

systems, and their state responsibilities overlap considerably 
with their SLR role. Centers, while having virtually no formal 
leverage over SLRs due to the SLRs 1 voluntary status, rely 

j 

L heavily upon these individuals in achieving their objectives. In 

fact, the perception of those involved in the Network is that the 
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degree 0 f personal commitment of SLRs to the goals of the Network 
is the single most critical variable affecting Network success. 

The SLRs are known as the "linkages" in the Network. They 
help shape Center policy by communicating their states 1 curri- 
culum-related needs to the regional Centers, while state and 
local personnel rely on SLRs for obtaining Center-provided 
materials and services. In a few states all requests for Center 
services are channeled through the SLR, although most state and 
local users are encouraged to contact the Center directly. The 
Centers rely on the SLRs to provide state- related information and 
materials, such as newly developed curricula. 

Center reporting requirements . The six Centers report to 
OVAE quarterly and annually on their activities and accomplish- 
ments. The format and substance of their impact reports have 
been the subject of considerable work by Center Directors over 
the years, owing in part to the problems inherent in tracking the 
utilization and impacts of dissemination and capacity-building 
activities and in part to the variability in the curriculum- 
related activities of individual states. Impacts are currently 
reported in "Key Results Areas" grouped under these broad 
activity categories : coordination and management, curriculum 
services, and dissemination and utilization. 

Because the reporting formats include many Key Results Areas 
associated with the Network's past priorities, accomplishments 
that occurred in past years continue to appear in each annual 
report (e.g., existence of state curriculum resource centers). 
An additional problem is the inconsistency of definitions and 
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tabulation principles used by SLRs as they volunteer state 
information for use by the Centers. Analysis of these and other 
issues concerning the Centers 1 reporting requirements is 
presented in Appendix D of this report. 

Study Methods 

In order to assess the impacts of the Network on users ot 
its services, the study team conducted a variety of data collec- 
tion activities, including a review of available program infor- 
mation, site visits to the Centers and other vocational education 
organizations, telephone interviews with vocational educators in 
a sample of states, and attendance at several Network/Center 
meetings. A brief description ot data collection and analysis 
activities is provided in this section. 

Analysis of available program information . This task 
involved two activities: (1) review and analysis of the 1984 
Impact Reports submitted by the Centers to OVAE and (2) review ot 
the Center files maintained by OVAE, which consist primarily ot 
Center proposals and related contractual and budgetary informa- 
tion. The Center impact reports were reviewed in order to iden- 
tify how program impacts are measured and reported and to assess 
the feasibility of reporting Center activities using the cate- 
gories of information resources, capacity building, and linkages/ 
outreach. Contract files maintained at OVAE were analyzed to 
obtain 1984 federal funding levels and to identify how the 
Centers budgeted these funds across major expenditure categories. 
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Site visits * Each of the six Centers was visited during 
either Phase I or Phase II of the study. The purpose ot the 
visits was to identify regional and Center-specific factors that 
might influence Center use and impacts. At each Center , inter- 
views were conducted with the Director and Center staff regarding 
all aspects of Center management and operations, including state- 
specific mechanisms used for accessing the Center and its 
resources. 

To examine fully the extent to which the Network has become 
an integral component of the nation 1 s dissemination and improve- 
ment system for vocational education , site visits were also 
conducted at NRCVE and V-TECS. In addition, a telephone inter- 
view was conducted with the Executive Director of MAVCC. 

Telephone 'interviews * Because of time and con 4 -, constraints, 
telephone interviews with Network users could only be conducted 
in a sample of states. A sample of 15 states was determined to be 
large enough to provide variation among states yet small enough 
to allow us to track the flow of Network resources and impacts at 
various user levels by interviewing several respondents in each 
state. 

A six-celled sampling frame was constructed along the dimen- 
sions of state curriculum practices, (producer or consumer) * and 
level of state vocational education resources and activities 



* Producers are those states that engage in curriculum 
development activities. Consumers purchase most of their curri- 
culum materials. 
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(high, medium, or low). Center Directors were then asked to 
categorize each of the states in their region along these two 
dimensions, and states were assigned to one of the six cells, as 
depicted in Table 1-1. 



Table 1-1 

Sampling Frame Used for Selection of States 
for Telephone Interviews 



State Level of Vocational 
Education Curriculum 
Resources/Activities 


Curriculum 
Producer 


Curriculum 
Consumer 


Total 


High 


n - 15 


n = 2 


17 


Medium 


n - 11 


n = 9 


20 


Low 


n = 0 


n = 1*4 


14 



Total 26 25 51 



Once all 50 states and the District of Columbia had been 
included in the sampling frame, we purposively selected from each 
region two or three states that varied in terms of the sampling 
criteria. Three states were selected from each of the Midwest, 
East Central, and Northwest Regions, and two states were selected 
from each of the three smaller regions. The final sample ot 
states resulted in the following configuration. 
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State Characteristics No, of States 

High Resources/Producers 4 

Medium Resources/Producers 3 

High K^source.VConsumecs 1 

Median* Resources/Consumers 3 

Low Resources/ Consumer _£ 

15 



The overall purpose of the telephone interviews was to ask 
vocational educators about the services they receive from their 
regional Centers. Thus questions were keyed to particular 
characteristics of each Center and each state selected within a 
Center's region. Telephone interviews were conducted in each of 
the sample states with the following categories of respondents: 

• State Directors of Vocational Education 

• State Liaison Representatives 

• State-level vocational educators 

• Local-level vocational educators 

• Teacher educators 

In each state, respondents were asked about the services they 
receive from their regional Center. Interviews with State 
Directors were designed to obtain their perceptions of the Center 
providing services to that state and the effects of those 
services as well as information concerning state vocational ed- 
ucation policy and practices that might influence those percep- 
tions. The areas covered in interviews with Center users 
included degree of familiarity with their SLR and regional 
Center, use of Center resources, Center impacts, and general 
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perceptions regarding Center/Network effectiveness. SLR inter- 
views addressed each of these areas in greater detail. 

Individuals interviewed at the state and local levels were 
randomly selected from a list of users whose names we obtained 
from the Centers and the SLRs. The study design called for 
interviews to be conducted with two state-level users, two local- 
level users and one teacher educator from each state. However, 
in one state, only one state-level user other than the SLR could 
be identified. In another state, none of several teacher educa- 
tors whose names were provided by the Center were still working 
in the state, and the SLR was unable to supply users in this 
category. Finally, in states where SLRs had been in that posi- 
tion for only a few months, we also interviewed the previous SLR. 
In sum, 103 telephone interviews were completed as follows: 



Respondent No. 
State Directors of Vocational Education 15 

State Liaison Representatives 15 

State-level users 29 

Local-level users 30 

Teacher educators 14 

Total 103 



Users at the state and local levels occupied a wide range ot 
positions. State-level users typically held supervisory or con- 
sultant positions in a vocational area such as home economics , 
business and cooperative education, industrial arts, health occu- 
pations, or agriculture. Also represented in the state-level 
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interviews were Research Coordinating Unit (RCU) Directors, a 
Resource Center Coordinator, a Research and Planning Section 
Chief, an assistant to the Deputy Director for Vocational Educa- 
tion, and specialists in such areas an interlibrary cooperation, 
sex equity, and high-risk youth, among others* Most of the local 
level respondents were county or area-wide Directors or Assistant 
Directors for Vocational Education* High school teachers, com- 
munity college deans or program directors , and Directors or 
Assistant Directors of Vocational-Technical schools constituted 
the remaining respondents* 

Network meetings * Members of the study team attended a 
meeting of the Directors' Council in early April 1985 to observe 
and collect information concerning the Council's role in coordi- 
nating Network activities. Study team members also attended the 
1984 Concurrent Meeting, the Regional Meeting of the Miawest 
Center in early April 1985, and the Joint Meeting ot the South- 
east and East Central Centers in late March 1985. These meetings 
allowed us to meet many of the SLRs from states in those regions 
and to gain first-hand knowledge of the nature of these forums 
for information sharing, capacity building, and coordinated 
planning • 

Data analysis * Data obtained through telephone interviews 
with Network users were analyzed to identify patterns of use, 
impacts, and perceptions of the Center/Network's utility across 
the various respondent categories. These data were examined in 
the light of contextual information associated with state policy 
and practice and various Center and regional characteristics 
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obtained through interviews with State Directors, site visits, 
and attendance at Network meetings* Broader impacts on the 
quality and effectiveness of dissemination and utilization in the 
vocational education curriculum system were examined largely 
within the context of the linkages the Network has established 
with related organizations* 

Based upon our initial assumptions regarding potentially 
important factors affecting patterns of Network impacts, as well 
as the results of initial data tabulations, several cross-tabula- 
tions of the data were performed in an attempt to identify rela- 
tionships between the three broad impact areas of information 
resources, capacity building, and linkages/outreach and certain 
variables associated with either the Centers or individual 
states. State characteristics examined as potential influences 
on how a state uses Center resources and how Center services 
affect state capacities included our initial sampling criteria of 
whether a state was a producer or a consumer of curriculum and 
the level of state resources and activity in vocational education 
curriculum* Other state factors we investigated included the 
presence of a state curriculum lab, membership in a curriculum 
* consortium, whether users contact the Center directly or through 

the SLR, whether the state is under a mandate to implement compe- 
tency-based curriculum, and a number of other state- related 
factors* We also examined the data to see if any regional varia- 
tions were evident and might be explained by Center- related 
characteristics such as funding levels, variation in services, or 

- 30 ^ 

; ERJC 



21- 



linkages with other organizations involved in vocational educa- 
tion curriculum. 

Organization of Report 

The findings and conclusions that emerged from the study's 
data collection and analysis are presented in the remaining chap- 
ters of this report. Chapter II presents the study's major find- 
ings concerning patterns of the Center/Network's activities and 
impacts in information resources/ capacity building, and linka- 
ges/outreach. Chapter III presents conclusions and recommen- 
dations based on the study's findings. Tables displaying select- 
ed data items from the SLR and state and local user interviews 
are presented in Appendices B and C, respectively. Appendix D 
includes an analysis of the Centers' impact reporting system. 
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II. STUDY FINDINGS 



The major study findings presented in this chapter are 
organized around Center/Network activity areas: information 
resources/ capacity building , and linkages/outreach. The primary 
data source for the section on information resources is the 
telephone interviews ; for capacity building , this source is 
supplemented by information from the Regional and Concurrent 
Meetings; for linkages/outreach, the data sources are interviews 
with other vocational education organizations and telephone 
interviews. Information obtained from users of Center resources 
was characterized by a high level of consistency and did not 
reflect variability along state or regional characteristics/ and 
other factors that were initially expected to influence the 
nature and extent of the program^ impacts. Consequently the 
study's findings are reported in the aggregate. 



Information Resources 

Goals of the Centers/Network include the provision of 
mechanises for (1) sharing information on curriculum materials 
and tne materials themselves, (2) increasing the effectiveness 
and transportability of these materials, and (3) avoiding 
duplication of effort by encouraging adoption and adaptation. 
The regional Centers address this goal through several means. 
They serve as information reposi tot ies or libraries. They 
disseminate to the. states and territories copies of materials 
that developing states wish to share. They support tools for 
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curriculum developers such as VECM and electronic mail and/ 
through meetings of their SLRs and contact with State Directors, 
promote the concept of adoption and adaptation. 

Regional Centers as information repositories . Have Centers 
developed useful repositories of curriculum materials and encour- 
aged the use of these materials? If they have, we would expect 
that SLRs who need curriculum materials would ( 1 ) reference 
curriculum developed by others rather than undertaking immediate 
development and (2) tap the resources made available through the 
Network's collection and dissemination of materials. We asked 
SLRs in our sample what procedures they typically follow when a 
need for specific curriculum materials arises in their state. In 
response to our open-ended question, all 15 SLRs told us they 
typically would begin by searching for existing materials/ thus 
indicating that the Centers have met one criterion ot success. 

Eleven of the 15 SLRs reported that their first or second 
step would be to contact their regional Center, Two ot the SLRs 
reporting a different sequence typically contact a consortium 
before their regional Center. Half of the SLRs reported that the 
first step they take when there is a need for curriculum materi- 
als is to review in-state resources. In most cases these SLRs go 
to a state resource center or curriculum development lab, housing 
among other things materials provided through the Network ; in 
other cases they themselves coordinate a repository ot Network- 
disseminated materials. Thus the Network provides support to the 
information sources to which these SLRs turn first. Table II-l 
summarizes the SLRs 1 rankings of sources. 
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Table II-l 

SLRs 1 Ranking of Sources Accessed in Meeting State 
Needs For Curriculum Materials (n - 15) 

Ranking* 

Sources 

State lab/center 
Regional CCC 
Consortium 
Another CCC 
NCRVE 

Another state/SLR 
Other 

State and local users mentioned a variety of sources other 
than the Centers for curriculum materials or services, but no 
other single source was mentioned by more than 24 percent of the 
users. Seven sources were each mentioned by between 14 and 24 
percent of users, including other states, consortia, in-state 
sources such as labs and resource centers, computerized data 
bases, colleges and universities, private organizations, and 
NCRVE. The diversity in these data strongly suggests the Network 
is serving a centralized access purpose not otherwise being met. 

Responses of SLRs indicate that there is nonetheless room 
for improvement in the use of the Centers 1 information resources. 
Asked if they believed their state made optimal use of available 
Center services, 10 of the 15 SLRs said they did not believe so. 



* Some SLRs cited less than 5 sources. 



JL JL jL A iL Total 

7 7 

5 6 2 13 

2 111 5 

1 1 2 

1 1 2 

3 3 

111 3 
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Reasons given included simply insufficient time to deal with the 
Centers on the part of SLRs (5 SLRs) and not enough awareness of 
the Center within the state (4 SLRs) . Only two SLRs (from dif- 
ferent regions) attributed their response to insufficient 
resources and capacity at their regional Centers. 

Thus the Network Centers have apparently established 
themselves as useful sources of information and materials. 
Fur the more , the materials available through them are generally 
adequate, based on the evidence we obtained. 

The emerging field of high technology was the only occupa- 
tional area that SLRs reported their own regional Centers do not 
adequately address. This field was mentioned by eight SLRs 
across five of the six regional Centers. It was the only inade- 
quacy mentioned for one regional Center, and this was the only 
Center for which the sampled SLRs expressed a consensus on a 
particular inadequacy. Four SLRs mentioned needs for more 
materials in more traditional areas such as e 1 ectronics , hotel/ 
motel , marine occupations , textile and leather manufacturing ; 
however, no single area was mentioned more than once , and 
(outside of high technology ) SLRs from the same Centers cited 
different needs. Similarly, we found no consensus among SLRs 
when we asked them about materials and services for special needs 
groups (e.g. , handicapped, limited' English proficient (LEP) ) . 
Only four perceived any inadequacy and each of these had- a dif- 
ferent experience. 

Other state and local users corroborated the evidence 
provided by SLRs. Eighty- three percent believed the materials 
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they had received through the Centers were useful r and 87 percent 
perceived the materials to have been of high quality. 

A basic premise for increasing the availability of informa- 
tion resources and encouraging adoption and adaptation is the 
concept of networking. A later section describes what the Net- 
work is doing to increase the capacity of curriculum developers 
to develop and use their own networks. Here we examine the 
Network channels used for accessing information. 

Nine SLRs make use of Centers outside their own regions by 
contacting them directly. The remaining six SLRs do not go to 
other regional Centers directly, but tap such Centers only 
through their own Centers. Thirteen SLRs contact other SLRs 
directly. Seven of these contact only SLRs in their own region, 
but in all but one region at least one SLR in the sample networks 
with SLRs outside his or her own region. 

The networking patterns among SLRs in our sample suggest 
that the regional assignments of states to Centers are appropri- 
ate. Only two SLRs reported that they exclusively contact SLRs 
in states outside their own regions , and no additional informa- 
tion from respondents in those states indicates dissatistaction 
with their states 1 assignment to particular regions. 

A general assumption is that users gain access to the Net- 
work through the SLR, although those familiar with the program 
agree there is variation in the emphasis placed on this avenue ot 
access. We found that one-half of the users in our sample 
reported they only access the Network through their SLRs. One- 
quarter always go directly to the regional Center, and one- 
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quarter use both methods of access. The geographically dispersed 
Western Center users— only one of whom reported contacting the 
Center directly — account for one-half of those accessing the 
Network only through the SLRs. In the other regions , two-thirds 
of the system users contact their Centers directly. Of the three 
categories of users ( state- level personnel , local-level person- 
nel, and teacher educators) , teacher educators are the ones most 
apt to make direct use of the regional Centers. Nine ot the 14 
teacher educators in our sample said they typically contact the 
Centers directly* 

Tools of curriculum developers . The Centers have taken 
steps to encourage and help vocational curriculum developers to 
use emerging technologies in carrying out their job. The concept 
of VECM originated within the Network, and the currency ,and 
comprehensiveness of this database result from the submission ot 
abstracts on products developed within the states to NCRVE via 
the Network. All of the states in our sample have the computer 
capability needed to access VECM directly. However, the vast 
majority of the SLRs reported continued reliance on their 
regional Centers or NCRVE to perform VECM searches. In general, 
searches, whether automated or paper, were the service that users 
most frequently reported obtaining from the Centet (48 percent ot 
users) . 

The regional Centers have also supported the concept ot 
electronic mail and have encouraged states to participate in 
ADVOCNET. Among the possibilities for future use of ADVOCNET are 
communications related to planning regional and national 



37 



meetings, obtaining information from states regarding curriculum, 
and making announcements to SLRs regarding Center activities. 

All but one of the states in our sample use the ADVOCNET 
electronic mail system. The single SLR not using ADVOCNET felt 
the regional Center provided sufficient information, thereby 
making use of the system unnecessary. Those who use ADVOCNET 
reported doing so because they find the system useful and 
timesaving • 

Adoption, adaptation, and quality . A goal of the Network is 
to encourage adoption and adaptation of curriculum materials in 
order to reduce duplication of effort. In practice, states and 
territories may vary in their ability and willingness to adopt 
and adapt due to several factors. For example, curriculum exists 
within the broader context of state policies, and state organiza- 
tional structure establishes the point of responsibility. State 
budgets dictate the resources that can be applied to curriculum 
development and improvement. 

Despite the variety that exists along these factors across 
the states and territories, we spoke with only one State Director 
who described a philosophy and practices that are not compatible 
with the Network's goals regarding adoption and adaptation. We 
found a high level of consensus among the 15 State Directors in 
our sample regarding the importance of the Centers to the states 1 
past and future ability to respond to curriculum needs . While 
several suggested ways in which the Network could better fit 
their specific needs , we found that the Network has managed to 
attain high utility to states despite state differences. 
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Six of the State Directors with whom we spoke described 
their regional Center as an integrated component of their 
curriculum program and reported that referral to the Center is a 
general practice when curriculum needs arise within the state. 
They told us they adopt or adapt curriculum materials whenever 
possible. Four gave us examples of how the Network has helped 
them with past curriculum needs. They reported instances where 
the state saved time and effort by building upon the work already 
accomplished by others. Two State Directors described only 
limited use of their Centers. 

Four of the State Directors whom we interviewed spoke of 
their concerns regarding the future. As vocational education 
moves into the 1990 f s and the 21st century, these State Directors 
believe sharing, adoption, and adaptation will facilitate the 
system's ability to respond to changing needs. They see the 
Centers as playing an important role in this process. 

The regional Centers disseminate curriculum materials such 
as curriculum guides, task lists, resource guides, instructional 
materials, and competency-based vocational education (CBVE) 
materials to vocational educators responsible for curriculum at 
state, regional, and local levels and to teacher educators. The 
users with whom we spoke use these materials most frequently in 
the preparation of new materials. One-half of our sample report- 
ed this application. One-third of the users reported that the 
materials they acquired through the Network were used for 
purposes of staff development and training of new teachers. 
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One-third of the users in the sample passed the materials on to 
others to use either in curriculum development or in training. 

SLRs most frequently cited the Network's dissemination 
function as the reason for their perception that the Network and 
its services have improved their state's vocational education 
curriculum. One-half of the sampled SLRs described the contribu- 
tion of increasing the pool of materials available to individual 
curriculum developers. One-quarter attributed the perceived 
effect to the improvement in their states' use ot existing 
materials. In all, reasons given for improvement by two-thirds 
of the SLRs were related to this one function of the Network. 



Capacity Building 

The term "capacity building," as used nere, reters to 
activities designed to result in either (1) the acquisiton ot 
specific skills, knowledge, or awareness of individual vocational 
educators or (2) the acquisition or improvement of statewide 
systems tor vocational educational curriculum development, dis- 
semination, and utilization. Network influences on the capaci- 
ties of specific individuals or state systems can be provided 
directly through meetings or the provision of information or 
technical assistance. The Network can also build capacity 
indirectly through the activities of* SLRs, who draw upon Center 
resources in building the capacities ot vocational educators in 
their state and in improving state capacities generally* 
Finally, Network impacts on individual or system capacities may 
be immediate in effect or may accrue over several years. 
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I" The Networ*. attempts to build individual and organizational 

capacity because increased capacity in the system is one of its 
objectives and also because it depends on the SLRs and others to 
advance all its objectives. SLRs are an integral part of the 
Network's means of sharing information on curriculum materials , 
determining needs for curriculum materials, and coordinating 
development, dissemination, and utilization. Network effective- 
ness requires that these individuals have the requisite skills 
and knowledge to fulfill their responsibilities. All but two ot 
the 15 SLRs with whom we spoke reported having major responsibil- 
ity for curriculum or program development activities in their 
states, and all but three have positions overseeing general voca- 
tional education (as opposed to some specific programmatic area 
such as trades and industry or career education) . Beyond the 
SLR, the Network's objectives are furthered when teacher educa- 
tors, program specialists , area coordinators, and other curri- 
culum developers undertake their work with an understanding ana 
acceptance of the principles the Network represents. 

Capacity building for the SLRs occurs primarily in three 
forums — at the Concurrent Meeting , at Regional Meetings , and 
through technical assistance. In the aggregate, 17 percent ot 
the Network ' s budget is earmarked for travel, primarily in 
support of these activities. The Network's major means of 
building the capacity of other vocational educators are the 
technical assistance provided by the Centers , and workshops and 
technical assistance provided by SLRs. We discuss first the 
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capacity-building activities aimed at SLRs and then those aimed 
at others. 

Capacity building for SLRs . Building the capacity of SLRs 
is a recurring theme within the Network. Not only do vocational 
education technologies and approaches change, but SLRs also 
change. The average SLR in our sample had been an SLR for tour 
years, and two of the 15 had been SLRs for one year or less. 
This rotating and changing of the individuals having SLR 
responsibilities lead the Network to continuously provide 
training in networking skills* It also has the unintended effect 
of "spreading the word" through the vocational education system 
as former SLRs move from one job to another and even from one 
state to another. 

The Concurrent Meeting is the major Networkwide forum for 
capacity building* It begins with a session for new SLRt on the 
responsibilities of SLRs and the resources available to them. 
During 1984, the Network undertook development of a competency- 
based training package for SLRs that will be used as the basis 
for this training in the future. 

Presentations and inservice opportunities at the Concurrent 
Meeting are held in general sessions and mini-sessions. At the 
1984 Concurrent Meeting , roughly 12 hours were dedicated to 
capacity building. Half of the capacity-building portion of the 
agenda was devoted to inservice in emerging technologies and new 
approaches to curriculum development, one-quarter dealt with 
networking skills, and one-quarter with national priorities. In 
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the evenings, SLRs could review products of commercial vendors 
and state-developed materials. 

The Center Directors routinely evaluate the Concurrent 
Meetings using a five-point Likert scale to ascertain parti- 
cipants 1 perceptions of the importance and utility ot the overall 
program and specific sessions. In 1984 , SLRs recorded a rating 
of 4.72 for importance and 4.50 for the utility of the overall 
program. SLRs 1 ratings for the utility of individual sessions 
ranged from a low of 2.50 to a high of 4.75. Averaging the 
figures across sessions, SLRs reported the following levels of 
utility for presentations in the major capacity-building areas: 

• 3.86 - national priorities and perspectives 

• 4.08 - networking skills 

• 4.06 - emerging technologies and new approaches 

The Concurrent Meeting also gives SLRs the opportunity to 
meet their colleagues from all the states and territories. 
However, some SLRs with whom we spoke perceived that they do not 
take full advantage of the networking opportunities available in 
this national forum, tending instead to spend unstructured time 
with the SLRs they know from their own region. These individuals 
believed that smaller meetings better foster the creation and 
cementing of new relationships. 

Concurrent Meetings are a contractually specified Network 
activity for the Centers. Each Center Director designates an SLR 
to represent the region on a planning committee which reports to 
the Director's Council. The role of host rotates among the 
Centers. Each Center is responsible for the travel expenses of 
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all SLRs in the region. The states cover the salaries of SLRs 
during their week away from regular duties while attending the 
Concurrent Meeting. Attendance figures for SLRs are high — a fact 
that not only validates the SLRs' relatively high evaluation of 
the utility of the meeting but also provides an indirect indica- 
tion of State Directors' perceptions of its utility. 

A second contractually described capacity-building activity 
of the Network is Regional Meetings. While all Network Centers 
are required to hold Regional Meetings , the number of meetings 
per year varies across Centers depending on the availability ot 
federal funds for the base contract year. At the time of this 
study four Centers were funded to convene three meetings a year; 
two were funded for two meetings. 

Each Center convenes one of the meetings in conjunction with 
the Concurrent Meeting where a day is set aside for regional 
sessions. These sessions typically include business items and 
sharing of new curriculum materials; capacity building occurs in 
other parts of the Concurrent Meeting agenda. 

In other Regional Meetings, the proportion of the agenda set 
aside for capacity building varies. During the most recent cycle 
of Regional Meetings, five of the six Centers offered presenta- 
tions on new or different approaches to vocational education 
curriculum; four held sessions intended to increase the SLRs 1 
networking skills; and two scheduled presentations on the new 
Carl Perkins Vocational Education Act. 

SLRs report that they bring useful skills and knowledge back 
to their states from the capacity building sessions at Regional 
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Meetings* Eleven of the 15 SLRs with whom we spoke had attended 
their region's latest meeting, while the other 4 had sent alter- 
nates. Ten of the 11 who attended a meeting reported having 
taken action on some new, specific information obtained from the 
Regional Meeting. Two had not debriefed their alternates at the 
time of our interviews and thus could not respond to our 
question. One had not yet had time to follow up on specifics 
from the Regional Meeting. One had planned the presentations and 
thus had not acquired new information. 

SLRs use what they learn at Regional Meetings in various 
ways. In some cases, according to our interviews, they brief 
.their State Directors on the information that might be useful for 
the state. Some have also arranged various types of follow-up 
events in their own states, including presentations for top state 
officials and workshops for local educators. Three SLRs in our 
sample reported having arranged workshops for vocational educa- 
tors on some aspect of curriculum development or use. 

Another major activity at Regional Meetings also builds the 
capacity of SLRs to link with their colleagues. This is the 
information-sharing portion of regional agendas, in which SLRs 
describe what their states are doing in curriculum development 
(in some instances circulating copies of products) and identify 
areas where their states need materials. Our observations 
indicate that these sessions contribute more to the Network than 
just sharing across states. They provide opportunities for each 
SLR to better understand the capabilities and resources of other 
states in the region. 



45 



-36- 



Assessments by SLRs , Center Directors, and others who attend 
the Regional Meetings also indicate that these meetings stimulate 
and facilitate linkage among curriculum developers. The meetings 
help personnel in one state to acquire the capacity to tap skills 
and curriculum resources in other states. The mechanism tor 
building this capacity is not exclusively formal. Linkage 
capacity is strengthened by conversations held over cof t ee, 
lunch , and dinner at Regional Meetings as SLRs with common 
vocational education interests seek one another out. The 
smaller, less structured environment ot single region and joint 
Regional Meetings was seen and reported to have an advantage over 
Concurrent Meetings in this regard. 

Summing up, when asked if their experience as an SLR had 
contributed to their professional development, the SLRs 
overwhelmingly responded that it had (14 of 15). They were 
unanimous in perceiving that their SLR experience had contributed 
to their effectiveness in their jobs. The reasons they gave for 
these assessments related directly to the Network's capacity- 
building efforts. Almost all reported their SLR experience had 
had these impacts by increasing their awareness of resources and 
developments in vocational education and by exposing them to new 
ideas. One-third of those in our sample reported that the 
Network-provided opportunity to interact with others in the field 
had contributed to their professional development. 

Capacity building for other users of the Network . The 
ability of the regional Centers to sponsor capacity building 
among users is constrained by the contractual requirements and 
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funds for technical assistance , although Centers can provide such 
services on a cost- reimbursement basis. During 1984 , there were 
32 Network-sponsored technical-assistance visits reported system- 
wide. The number of visits per Center ranged from a high ot nine 
to a low of two during that year. 

Forty percent of the users with whom we spoke reported 
receiving curriculum- related capacity-building services either 
directly from their region's Center or through their SLR. 
Proportionally more users at the local level reported receiving 
capacity-building services than did those at the state level or 
the teacher educators in our sample (Table II-2). 

Table II-2 

Number of Network Users Receiving Capacity-Building Services 



Capacity-Building Services Users 







Local State 


TE Total 


Inservice or workshop 




11 


4 


3 18 


Conference/presentation 




1 


3 


1 4 


Consultation 




2 


2 


4 


Technical assistance 






2 


1 3 


None reported/received 




16 


17 


9_ _43 


Total 




30 


28 


14 72 


Tho se who re ce i ved 


these services unanimously reported that 


they were indeed useful 




Reasons given to explain the ways in 


which capacity building 


was useful included increased awareness, 


greater staff development , 


an aid in 


curriculum 


development, and 


a means to stay current 


in 


vocational 


education 


developments . 



One intent of the Network's effort dedicated to building SLR 
capacity is to create momentum for building capacity throughout 
the vocational education curriculum system. While we did find 
evidence that this is occurring, it is not highly organized and 
thus not easy to track or measure. This may be an area in which 
there is room for improvement in the Network's effectiveness. 

At another level are the indirect, longer term influences 
that have accrued to statewide vocational education systems as a 
result of Network-provided services. For instance, as discussed 
earlier in this chapter, the role of the Network in stimulating 
the establishment of state curriculum laboratories and resource 
centers and in providing curriculum materials for these centers 
from curriculum producing states has resulted in an increased 
capacity in many states to satisfy their curriculum needs using 
state-held resources. 

Linkages and Outreach 

Since the program 1 s inception, the Network and individual 
Centers have developed formal and ad hoc relationships with other 
organizations involved in vocational education curriculum and 
program improvement. Such linkages have facilitated the sharing 
of materials and resources and have served as a vehicle for 
identifying and responding to emerging trends in vocational 
education. Linkages can also serve an important outreach 
function, helping persons in the field to become aware ot the 
information and other resources the Network offers. Further, in 
addition to the outreach function served by relationships with 
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other organizations , each Center and the Network as a whole have 
performed a variety of outreach activities, often using the SLRs 
as a vehicle for informing state and local vocational educators 
about the availability of Network resources. In this section, we 
describe the nature and effects of these linkage and outreach 
activities. Most of the information reported on linkages is 
derived from interviews with persons in other vocational educa- 
tion organizations • For OuLifedCxi activities, this source is 
supplemented by data from the telephone interviews. 

Linkages 

MAVCC . One of the first linkages developed after the 
establishment of the Network was the creation of a new vocational 
organization, which has subsequently been closely aligned with 
one of the regional Centers. At the first Regional Meeting held 
by the Midwest Center, SLRs in that region's states became aware 
of the amount of duplication in curriculum development activities 
occurring across their states. They decided to pool their 
resources in order to eliminate duplication and spend their 
curriculum funds more efficiently. MAVCC was established in 1975 
and has maintained close informal ties with the Network since 
that time . While the Network reportedly serves as a system for 
communicating national perspectives within the region, MAVCC 
focuses on developing curriculum materials for its member states. 

According to MAVCC 1 s Director, while there is no formal 
relationship between the consortium and the Midwest Center or the 
Network as a whole , operationally there are a number of connec- 
tions. The fact that most of the consortium 1 s state board 
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members are also their states* SLRs facilitates the flow of 
information regarding curriculum and other topics into MAVCC* 
The consortium often piggybacks meetings on Midwest or Northwest 
Regional Meetings, an arrangement which further promotes communi- 
cation on development among and beyond member states. 

MAVCC also uses the Network's information resources in its 
curriculum development activities. For new projects, consortium 
staff always conduct searches through the Midwest Center, and 
even if the searches do not uncover directly useful materials, 
some of the resulting references are appended to curriculum 
materials for use by local teachers as resource documents. The 
consortium has not needed to develop its own library because 
MAVCC curriculum developers use the Midwest Center's library. 

The consortium also works with other regional Centers on 
curriculum-related activities. Consortium staff have provided 
inservice workshops under the auspices of the Northwest Center, 
and they recently worked with the East Central Network to develop 
an inventory of the distribution of MAVCC materials and provision 
of inservice to states in that region. 

V-TECS . According to the Executive Director of V-TECS, 
while there is no formal relationship between the consortium and 
the Network, a substantial amount of sharing between the two 
organizations does occur. The V-TECS Director typically attends 
at least some of the Network's Regional Meetings each year, where 
he reports on current V-TECS activities. Additionally, one or 
more of tha Centers work with V-TECS on special projects. For 
example, the East Central Center recently assisted V-TECS to 
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analyze options for changes in membership requirements and struc- 
tures. Further^ although in the past V-TECS did not routinely 
share V-TECS-produced materials with the Centers , in recent years 
this pattern has changed , and vocational educators now have 
access through the Centers to V-TECS materials for preview and as 
resources for curriculum development or adaptation. The V-TECS 
states also routinely access Network resources as a first step 
when they develop V-TECS materials. The fact that state V-TECS 
coordinators have also been SLRs has facilitated a sharing of 
knowledge and expertise across the two systems. 

NCRVE . The Network has an ongoing , formalized relationship 
with NCRVE that in recent years has resulted in a number of joint 
projects and initiatives. Among the activities on which the two 
programs collaborate are the VECM database, ADVOCNET, dissemina- 
tion of NCRVE materials through the SLRs, SLR training sessions 
at the annual Concurrent Meetings, and a number of outreach 
activities . 

VECM/ an automated curriculum materials system that includes 
abstracts of courseware as well as hard-copy materials / was con- 
ceived and initiated by the Center Directors as an efficient 
means for persons undertaking curriculum development/ adoption or 
adaptation to scan available resources. It does not replace the 
libraries of the Centers and states because it does not provide 
copies of materials for preview or adoption, and it does not 
include commercially produced curriculum/ but it facilitates 
identification of those materials likely to be most directly 
relevant to a particular need. 
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VECM is the joint responsibility of the Network and NCRVE. 
While the regional Centers vary somewhat in VECM procedures , in 
general SLRs are asked to complete abstracts on materials 
developed in their states. The Centers edit these abstracts and 
transmit them to NCRVE for entry into the system. The database 
is kept up to date because the regional Centers arrange for 
materials to be deleted as they become obsolete. 

According to interviews conducted during the study , the use 
of VECM has increased in the last year or so. The database is 
accessed by private sector organizations, libraries, the mili- 
tary, and other organizations in addition to SLRs and regional 
Cente rs. 

According to the NCRVE official in charge of VECM, the 
organization of the system as a joint Network/NCRVE activity is 
important to its operation. NCRVE manages the system, receiving, 
editing, and inputting abstracts as well as providing ongoing 
consultation and assistance to the field. The role ot the 
Network is structured to minimize duplication in that each 
regional Center is responsible only for materials developed by 
states in the region. Because they have close relationships with 
the state and local vocational educators in their regions, 
primarily through the vehicle of the SLRs, the Centers are better 
able than NCRVE to facilitate the flow of materials into the 
system. Moreover, when gaps are identified, as in curriculum for 
special needs groups or vocational education courseware/ the 
Centers can target these areas for special emphasis and ask the 
SLRs to canvass states for materials. Finally, an additional 
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benefit of the current VFCM structure is that the state personnel 
who submit materials gain a sense of ownership of the system that 
seems to make them more likely to use it. 

In addition to management of VECM, NCRVE is also responsible 
for managing ADVOCNET, the electronic mail system for vocational 
education. Currently all six regional Centers participate in the 
system and are assisting NCRVE in its attempts to enlist voca- 
tional educators throughout the country in the system by provid- 
ing consultation and technical assistance on the system to SLRs. 

Another joint initiative recently implemented between the 
Network and NCRVE has been an arrangement recommended by the 
Director of one of the regional Centers under which NCRVE 1 s 
Dissemination and Utilization division share NCRVE materials with 
SLRs. Und^r this arrangment, NCRVE provides free materials to 
the Centers and participating SLRs, who then display the 
materials at conferences and workshops. NCRVE staff have 
developed a display notebook for SLR use that includes sections 
on Network and state materials as well as on NCRVE materials. 
This initiative is viewed as a two-way street: the activity 
helps NCRVE to reach state vocational educators who are part ot 
their target audience , and SLRs 1 resources are expanded through 
receipt of materials developed at NCRVE. 

In addition to such special projects as VECM, ADVOCNET, and 
the dissemination initiative, a number of other sharing activi- 
ties routinely occur between the Network and NCRVE. NCRVE 1 s 
officially designed Network liaison provides workshops for SLRs 
at the Network's Concurrent Meeting. NCRVE 's biannual news- 
letter, Vocational Educator, has an "SLR Corner" that profiles 
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individual SLRs and describes activities of the Centers and 
SLRs. These and other cooperative activities are driven by the 
NCRVE perception of the Network and SLRs as an integral component 
of the nation's vocational education dissemination and utiliza- 
tion system. 

Outreach 

Effective outreach is an important ingredient in the 
Centers/Network's success in achieving its objectives. £mong the 
vehicles for reaching potential users of Network services are the 
orgar izational linkages wich other consortia and agencies. Thus, 
for example, NCRVE includes information about the Network in its 
own dissemination activities, with the result that both programs 
increase their capacity to reach their target audiences. 
(Similarly, the SLRs constitute an important resource for NCRVE 
as a mechanism for communication with the states.) 

Another outreach vehicle used by the Network is the Reg ional 
Communiques that some Centers are contractually authorized to 
produce. Distribution of these and other information.! materials 
on the Centers 1 services helps to inform state and local 
vocational educators of the types of resources the Centers have 
to offer. For example, some Centers have printed brochures 
announcing their services, although such documents cannot be 
produced under the federal contract. 

The SLRs are probably the single most important outreach 

i 

w mechanism for marketing Network services. They communicate word 

about the Centers/Network through displays at conferences, 

u 
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presentations at workshops , and other mechanisms. As shown in 
Table II-3, local and statewide meetings and workshops are the 
avenues SLRs most frequently use to inform their constituents 
about Network resources. Results of our telephone survey of 
users of the Network indicated a high level of awareness of the 
regional Centers, with 88 percent of all respondents knowing 
specifically of their regional Center. (This finding exceeds the 
expectations of Center Directors , who tended to think users would 

Table II-3 

Mechanisms Used by SLRs in Performing Outreach 
to State and Local Users 

Type of Outreach Number of SLRs* 

Local meetings/workshops 12 

State-wide conferences/meetings 7 

CCC brochures 8 

State publication 6 

Through use of state lab/center 4 

Regional communique 1 

Association meetings 1 

know their SLRs by name but might not be familiar with the 
Centers or the Network.) As da' a reported in Table II-4 show, 
the SLR was the single most frequently mentioned source of 
awareness of the Centers. Further/ personal contacts appear to 
be a more useful outreach mechanism than publications, a finding 
that emphasizes the centrality of the networking function to the 
program^ effectiveness. 



* More than one response was provided by some SLRs 
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Table II-4 
Means of User Awareness of the Centers 

Local State T.E. Total 

(n * 23) (n = 27) (n ^ 14) (n ; 64) 

Means of Awareness 

Through SLR 4 12 1 17 (27%) 

State lab/center 1 2 1 4 (6%) 

State publications 3 1 3 7 (11%) 

State position 2 9 2 13 (20%) 

Presentations/meetings 3 1 4 (6%) 

Through someone other 

than SLR 6 1-7 (11%) 

Directly through Center - 1 12 (3%) 

Do not recall 4 - 6 10 (16%) 

Information presented in Table II-5 indicates the target 
groups addressed by SLR outreach activities, showing that local 
administrators and teachers are most frequently assigned the 
highest priority for SLR outreach activities. Other frequently 
mentioned targets of SLR outreach activities include state per- 
sonnel and postsecondary institutions. The priority assigned to 
postsecondary vocational educators may reflect a response to a 
criticism that in the past the Network has been less successful 
in reaching postsecondary vocational educators than those 
involved in secondary vocational education. Interviews with 
regional Center Directors, officials of the various consortia and 
NCRVE, and others identified postsecondary vocational education 
as an area that needs Concentrated outreach by the Network. 
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Table II-5 

SLR Outreach Priorities by User Categories 



Priority Rankings* 

User Categories 1. 2_ 3. ± Total 

LEAs 9 4 2 - 15 

SEAS 6 7 2 15 

Postsecondary 7 5 11 14 

Industry 2 1 3 



Finally, while our data collection activities indicated a 
high level of awareness of the Network among the various 
respondent groups, including State Directors of Vocational Educa- 
tion, state and local vocational educators, and teacher educa- 
tors, the continuing need for effective outreach is demonstrated 
by the fact that 28 percent of telephone interview respondents, 
asked an open-ended question regarding recommendations tor 
improvement of the program, specified increased outreach 
initiatives as their main recommendation. This finding suggests 
that, while current users of the Network's services are convinced 
of the utility of the services they receive, there is a wide- 
spread perception that some vocational educators, particularly at 
the postsecondary and local levels, need more information about 
the Network in order to access its resources effectively. 



* Some SLRs assigned the same priority ranking to more than 
one user category - 2 SLRs ranked first three categories as 
receiving equal priority. 
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Summary of Findings 

In general , the Network was reported to have affected voca- 
tional education curriculum activities at two levels. For state 
departments of vocational education, the Network has been exten- 
sively used and has had the effect of stimulating a level of 
cummunications among states that simply did not exist prior to 
the program's inception. At the local level, teachers have 
accessed Center-provided materials through their state labs and 
centers. These two channels were cited as having had a number of 
impacts, including : 

• Direct and indirect reduction of duplication in 
curriculum activities through stimulating communi- 
cation across the country 

• An increase in capacity in curriculum through serv- . 
ing as a vehicle for inservice training on curricu- 
lum, both indirectly through the SLRs, who receive 
inservice and then disseminate new knowledge through 
their states, and directly to local vocational 
educators through training in CBVE, curriculum 
development strategies, skills training in computer- 
ized databases, electronic mail, and other areas 

• Improvement in the overall quality of vocational 
education curriculum through helping people to 
become better informed regarding the range of 
materials available and to learn how to evaluate 
curriculum materials 

Variability in perceptions about the program's effectiveness 

hinges to some extent on states. 1 expectations about their 

regional Centers. At the same time, the key to the Network's 

effectiveness is the flexibility of the regional Centers in 

responding to the varied needs of individual states in their 

regions. While the Centers can fill a broad role in vocational 

curriculum activities, to be effective they must continue to 
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address the individual needs of the states within their regions. 
So it is critical that the states not get "defined out" ot the 
program as increasing emphasis is placed on national priorities 
and needs. 

The Centers have facilitated the spread of competency-based 
vocational education through promotion of this method in 
curriculum development, adoption, and adaptation. The wider 
availability of curriculum materials has reportedly improved the 
overall guality of vocational curriculum, although the program 
could have done more in this regard through some mechanism for 
endorsing the highest quality materials that are produced. 

Perhaps the most important benefit of the program has been 
the creation of a network that brings people together to learn 
from each other, take ideas and materials back to their states, 
and thus reduce duplication of effort in curriculum activities. 
In this context, the Regional Meetings have been a welcomed forum 
for sharing of ideas and materials. At another level, one ot the 
frequently overlooked benefits of the program has been the 
informal spread of expertise and resources through the movement 
of vocational education professionals around the country. As 
people move from one position to another within or across states, 
they carry with them the capacity and resources acquired through 
involvement with the Network, primarily as SLRs. Again , this 
spread of advocacy for the Network has increased the extent to 
which Network objectives have been accomplished . The continued 
focus of the program should revolve around these activities : 
collection, housing , cataloguing , and disseminating the latest 
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curriculum materials from throughout the country; and concentrat- 
ing on the networking relationships that facilitate sharing of 
materials, information, and expertise. The role of the Centers 
as a resource link is central, and because of limited resources, 
the Centers should concentrate on activities related to this role 
rather than attempting to "be too many things to too many 
people." 
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III. CONCLUSIONS AND RECOMMENDATIONS 

As documented in the preceding chapter, the findings ot this 
study suggest the Centers and Network are successfully achieving 
their objectives in the areas of increased availability ot curri- 
culum information and materials, reduced duplication ot curri- 
culum activities through promotion of adoption and adaptation, 
improved curriculum and dissemination services to state and local 
vocational educators , and linkages with other vocational educa- 
tion organizations. In this chapter we provide some conclusions 
about the program that emerge from the study's findings* Addi- 
tionally, a number of recommendations for consideration by the 
Centers and by OVAE in the context of planning tor the program's 
future direction are presented. 

Conclusions 

At the most fundamental level, a major objective ot federal 
officials in establishing the National Network for Curriculum 
Coordination in Vocational and Technical Education was to create 
a structure through which curriculum materials and resources 
would be readily available to vocational education practition- 
ers—curriculum developers, teachers, and others who need such 
resources* At this level the six regional Centers were intended 
to serve as resource libraries where materials could be accumu- 
lated and then distributed to persons engaged in curriculum- 
related activities throughout the country. The findings ot this 
study suggest that this program purpose has been. accomplished. 
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There is a high level of consensus among study respondents that 
the ready availability of materials at the Centers has enabled 
states to save time and money in their curriculum- related 
activities, as well as to reduce duplication ot effort in 
developing, adopting, or adapting materials for use in vocational 
education instructional activities. As one example ot this 
impact, two of the major curriculum consortia, V-TECS and JAVCC, 
both reported using Center resources in the curriculum 
development activities conducted by consortium members or statt. 

Further, the availability of this resource has apparently 
had some influence on decisions in a number of our sample states 
to reduce developmental efforts and move in the direction ot 
adoption and adaptation strategies where possible. Although 
other factors have clearly entered into such decisions (e.g., 
declines in resources available for curriculum and other program 
improvement activities, rapidly changing industrial conditions), 
the availability of the resource libraries has enabled states to 
look to curriculum development as an area where economies can be 
achieved. 

In addition to the resource library function ot the Centers 
themselves, the Network's strategy of encouraging states to share 
sufficient materials for distribution to all SLRs has created a 
pool of curriculum resources and materials in the states. (It 
should be noted that these materials have been contributed by the 
developing states and thus have not imoosed an added cost on the 
program.) Some states have recently begun to share materials in 
this way, although others have curtailed the number ot copies ot 
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materials they provide. The Center directors have continued to 
encourage states in their regions to share resources within 
constraints of state policies and practices. 

A number of respondents described a continuing and even 
accelerating need for the Network's information resources capa- 
city in the future. Specific factors cited as dictating the 
continuing need for the program include state mandates for compe- 
tency-based approaches to instruction, state-level educational 
reform acts, and state economic development initiatives designed 
to attract high-technology industry. More generally, rapid 
industrial and technological changes are straining states* capa- 
city to keep up with new curriculum needs, and one State Director 
mentioned that the Network will have an important role in taking 
vocational education into the 21st century. Finally, the current 
nationwide emphasis on educational excellence suggests a conti- 
nuing, need for the types of resources the Network helps to make 
available to the states. In this regard, it should be noted that 
accumulation and dissemination of curriculum materials is a 
continuing process — that is, the production of curriculum and 
instructional material is not time limited and this need tor the 
information resources through the Network is ongoing. 

While there is considerable variability in the activities 
and foci of the individual Centers, they are all repositories and 
disseminators of curriculum and instructional materials and serve 
as a linking agent to materials available in other states, at 
vocational education organizations, and in the national curri- 
culum database. Many of the persons interviewed tor this study 
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reported a perception that with these activities the program 
meets an important, ongoing need in vocational education. 

Beyond materials availability, perhaps the chief benefit of 
the Network documented by this study is the creation of what one 
respondent, who has worked with the Network for many years, 
termed "an intensive and highly capacitated communications net- 
work" across che nation. This communications network encompasses 

♦ 

the Centers and the current and former state-level SLRs. As the 
data reported in earlier sections of this report suggest, this 
communications network is extensively accessed nox. only by the 
Centers and persons working directly in activities related to 
curriculum and instruction but also by other vocational education 
improvement organizations. 

A high proportion of the federal resources available to the 
program has been devoted to the establishment and maintenance of 
this nationwide network. National and Regional Meetings, tech- 
nical assistance and consultation, Regional Communiques and elec- 
tronic mail have been used to create a climate in which there can 
exist in each state a locus of activity for interstate and intra- 
state sharing of knowledge and materials on vocational education 
curriculum. According to nearly all of our study respondents, 
this network operates efficiently and ettectively. SLRs 
routinely communicate with each other, with their regional and 
other Centers in the Network, and with other resource organiza- 
tions to locate the resources needed by vocational educators in 
their states. 
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The federal investment in the program has leveraged a 
considerable amount of state support. All but one of the Centers 
receive support from the states in which they are located , and 
several Centers receive substantial state funding for statt , 
materials , and activities , expanding their capacity to provide 
services to their own state and other states within and beyond 
their regions. Further, all states and territories contribute to 
the Network by donating the time of the SLRs to attend to the 
Network's business. Finally, many of the states that expend 
considerable resources in curriculum development share their 
products with the Network, thus enabling states with lower 
resource levels to obtain materials that would otherwise be 
unavailable to them. One of the study's respondents, the 
director of a curriculum consortium, commented regarding the 
leveraging of resources, that "if you could document it, the 
federal investment has been some ot the best money spent in 
vocational education." 



Recommendations 

Our data collection and analysis uncovered several areas in 
( which the regional Centers, the Directors 1 Council, or OVAE might 

* consider adjustments to the program that could enhance its effec- 

tiveness. Such adjustments are offered for consideration in the 

4 

context of a program that appears to be operating largely as 
\ intended. The recommendations fall into two categories: (1) 

actions that might be implemented by the Centers or Network 
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through the Directors' Council and (2) actions that might be 
undertaken by OVAE in its management of the program. 

Evaluation of curriculum . The existence of the Network over 
the past decade and a half is generally thought to have improved 
the quality of vocational education curriculum. The perception 
is that increased exposure to curricular and instructional 
materials and technical assistance in curriculum development and 
applications have increased the capacity ot practitioners to 
recognize and select high quality materials. It might be pos- 
sible for the Network to do more, however. While the essentially 
voluntary nature of the Network is thought by many to prevent the 
Centers from evaluating the materials they obtain from the 
states, several respondents recommended that the Network under- 
take some sort of curriculum evaluation. One suggestion that 
might be feasible within the context of state sharing ot mate- 
rials is a review procedure that would not exclude materials trom 
the resource base but would provide endorsements of materials 
that met certain standards of quality (e.g., materials that had 
undergone field testing and validation ) . 

Outreach . A second observation about which there was 
consensus among study respondents was the need tor increased 
outreach. While the Centers are- limited in their outreach 
activities by printing restrictions, SLRs could be encouraged to 
perform more outreach activities, perhaps through state publica- 
tions. The Centers might want to provide training in outreach 
strategies in their Regional Meetings and add a section on out- 
reach to the training package used with new SLRs. Additionally , 
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a number of respondents recommended targeting of outreach to 
groups that have in the past underutilized Network resources , 
particularly postsecondary vocational education institutions. 

Joint meetings . The Regional Meetings are generally viewed 
as an extremely important forum for information sharing and capa- 
city building among SLRs. While the annual Concurrent Meeting 
was reported by some respondents to be too large to permit some 
of the most useful sharing and capacity-building activities that 
occur at Regional Meetings, one problem mentioned concerning 
Regional Meetings is that sometimes they are less effective than 
they might be, particularly for the smaller regions when some 
SLRs are unable to attend. In part for this reason, and in part 
because persons who have attended some of the recent joint 
meetings have been very enthusiastic about what they have learned 
about activities in other regions, one alternative might be an 
increase in the use of joint meetings. For example, Centers 
might co-sponsor a Regional Meeting every other year on some type 
of rotating basis. 

Selection of SLRs . SLRs are appointed by their State Direc- 
tors of Vocational Education, and the time they spend on Network 
activities is donated by their states. In general, this practice 
appears to have worked well. At the same time, the importance of 
the SLRs to the effectiveness of the program, along with such 
recent program developments as the advent of VECM and electronic 
mail , suggests the appropriateness of the Center Directors 1 
involvement in the selection process. For example, the Direc- 
tors 1 Council might develop a listing of Network activities that 
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affect SLR responsibilities and time commitments for review by 
State Directors as they make their appointments. 

Impact reporting . As currently ' structured, the Center 
impact reports provide a wealth of historical information, 
process data, and impact information keyed' to both state and 
Center activities (see Appendix D for an analysis of the current 
system) . To provide a clearer picture of the impacts of the 
Centers and states in their regions in accomplishing program 
objectives, a critique and revision of the impact reporting 
system by the Directors 1 Council might be useful. Such an 
activity might focus on distinguishing between the Centers/ 
Network's management information needs and the types ot measures 
that pertain more specifically to Center and Network impacts. 
Among the revisions the Council might consider are the following: 

(1) Ensuring uniform interpretation cf the meaning of 
individual impact measures ana consistency in 
methods used to calculate the numbers reported. 

(2) Separating noncomparable activities now incluaed 
under a single "result area," such as "Items 
disseminated , " which may include outreach 
materials in addition to curriculum- related 
materials, thereby confounding analysis ot Center 
impacts in specific areas. 

(3 ) Distinguishing between activities and impacts ot 
the Center and those of SLRs. now lumped together 
in several measures such as number ot inservice 
workshops held. 

(4) Requiring all Centers to provide impact data on a 
state-by-state basis, in order to facilitate 
assessment of the effects individual states have 
on overall regional patterns of activity. 

(5) Deleting several static measures from the impact 
reports, particularly those relating to the 
presence of state systems tor vocational education 
curriculum, such as library loan systems, and 
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those relating to standard SLR functions such as 
attendance at regional meetings and maintenance ot 
communications with related state staff. 

(6) Incorporating into the reports several new 
measures designed to assess the degree to which 
various technology-based improvement efforts are 
being implemented , such as use of ADVOCNET by 
client states , states performing their own VECM 
searches/ and percent or number of Center 
materials accessible on-line. 

Consideration of these and any other improvements to impact 
reporting must be done with the understanding that SLRs provide 
state-related data to the Centers on a voluntary basis. Centers 
are therefore limited in their ability to prescribe data collec- 
tion or analysis. 

In addition to adjustments that might be undertaken by the 
regional Centers, several recommendations emerged concerning 
OVAE's management of the Network. These pertain to budgetary 
planning, concentration in emerging areas, and reconsideration o£ 
the current regional configuration of the Centers. 

Budgetary planning . Federal management ot the Network is 
constrained by the program's dl jcretionary nature. In years when' 
f eW er resources are available, some activities are deleted from 
Center budgets. Among the activities that are typically used to 
adjust budgets to available resources are technical assistance. 
Regional Meetings, and Regional Communiques , presumably because 
each is a discrete activity that can be added or deleted without 
implications for overall Center operations. 

While the fluctuations in federal funding may be unavoid- 
able, longer range strategies might be considered as a way to 
alleviate the operational problems caused for the Centers. For 
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example , currently the Western Center, which has relatively more 
communication problems than other Centers because ot the large 
geographical area it serves, does not have funding for a Regional 
Communique . Additionally, since nearly all study respondents 
cited the Regional Meetings as the single most useful Network 
activity in terms of information sharing and capacity building, 
alternatives for cost savings other than reduction in the number 
of Regional Meetings should probably be identified. 

Concentration in emerging areas . The areas of information 
resources cited as needing more attention by the Nccwork are such 
emerging areas as curriculum for high technology fields (e .g . , 
robotics and computer-assisted design and machining) and instruc- 
tional software. Given the rapid change? in these fields and the 
expense associated with them, eacn Center might take one emerging 
area as a special concentration in addition to the current 
library services offered by all Centers. This strategy would 
provide the nation with a specified place for obtaining needed 
materials in each high-cost area of concentration. 

Such an arrangement would likely require a somewhat more 
formal role for the Directors 1 Council because greater coordina- 
tion would be required in the decisionmaking process. However, 
the benefits might justify whatever minor changes would be 
required. 

Changes in the regional* configuration of the Centers. A 
number of respondents had recommendations concerning the current 
regional configuration of the Centers. While the regional 
structure of the prog ram appears to be one of the factors that 
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has contributed to its success/ particularly in establishing 
communications and rapport among SLRs/ some respondents pointed 
to specific variations among states within regions that atfect 
the utility of some of the information sharing that occurs. Some 
states are unlike the rest of their regions in such important 
characteristics as industrial profiles, availability ot resources 
for vocational education, and demographic/geographic characteris- 
tics (particularly in the case ot states that are large but 
sparsely populated and have relatively low resource levels). At 
the same time, in general the SLRs interviewed in the study 
reported a high regional affiliation, and thus the current 
regional configuration may be the most appropriate one. 

Summary , Although the effectiveness of the program 
constitutes an argument against major changes at the Center or 
federal level, the Network is a mature program, having been in 
place since 1972. Given current and expected changes in national 
industrial and employment patterns, it may be useful tor program 
managers and operators to think about where the program has been 
and whether some new directions might be usefully considered. It 
is clear that vocational educators who provided information tor 
this study see a continuing need for the program, and it is 
within this context that consideration ot possible future 
directions may be appropriate. 
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APPENDIX A 

REGIONS, STATES SERVED, AND ORGANIZATIONS 
OPERATING EACH CENTER 



Northeast Region 



Southeast Region 



Midwest Region 



East Central Region 



Northwestern Region 



Western Region 



Connecticut, Maine, 
• Massachusetts , New 
Hampshire, New Jersey 
New York, Puerto Rico, 
Rhode Island, Vermont, 

Alabama, Florida, 
Georgia , Kentucky , 
Mississippi, North 
Carolina, South 
Carolina , Tennessee 

Arkansas, Iowa, Kansas, 
Louisiana, Missouri , 
Nebraska, ' New Mexico, 
Oklahoma, Texas 



Delaware, District of 
Columbia, Illinois, 
Indiana , Maryland, 
Michigan, Minnesota, 
Ohio, Pennsylvania, 
Virginia, West Virginia, 
Wiscons in 

Alaska , Colorado , 
Idaho, Montana , North 
Dakota , Oregon , South 
Dakota, Utah, Washing- 
ton, Wyoming 

American Samoa, Ari- 
zona , California , 
Commonwealth ot the 
Northern Marianas, 
Guam , Hawa i i , Nevada , 
Trust Territory ot 
the Pacific Islands 



New Jersey 
State Depart- 
ment of 
Education, 
Old Bridge, NJ 

Mississippi 
State Univer- 
sity, 

Starkville, MS 



Oklahoma State 
Department ot 
Vocational and 
Technical Edu- 
cation, 

Stillwater, OK 

Illinois State 
Board ot 
Education 
Springfield, IL 



Northwest 
Futures , 
Glympia, WA 



University ot 
Hawaii, 
Honolulu, HI 
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APPENDICES B AND C 
SELECTED DATA TABULATIONS 



Tabulations of responses for selected items from the SLR and 
' state and local user interviews are provided in Appendices B and 

C, respectively. Simple counts have been used to express 
response frequencies in the SLR tables, since percentages based 
upon such a small number of respondents (15) would be misleading. 
Response frequencies for the state and local user interviews are 
provided by category of user. User categories and the total 
number of respondents in each category are as follows: 

User Categories No. ot Respondents 
Local-level users 30 
State-level users 29 
Teacher educator (T.E.) 14 
Total 73 

Response frequencies across all categories of users are expressed 
both as a count and as a percentage of the total number ot 

f respondents. 

* 

These frequency distributions are intended merely to eluci- 
t date relevant portions of the text and to provide the reader with 

some sense of how users responded to specific interview items. 
*' For many of the tables we collapsed similar or closely-related 

responses into a single category in order to develop a broad 
L perspective of user perceptions about the Network and its 

services. Explanatory notes to facilitate understanding ot the 

data are provided where necessary. 
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APPENDIX B 

TABULATIONS OF SELECTED ITEMS PROM INTERVIEWS WITH SLRS 



Table B-l: 
Table B-2: 

Table B-3 : 

Table B-4 : 

Table B-5: 

Table B-6: 

Table B-7: 
Table B-8: 

Table B-9: 

Table B-10: 

Table B-ll: 
Table B-12: 

Table B-13: 
Table B-I4: 



Page 

SLRs 1 State Responsibilities B-3 

Configuration of Curriculum Resources in B-4 
Sample States 

SLRs* Ranking of Sources Accessed in B-5 

Meeting State Needs for Curriculum 

Materials 

Mechanisms in Place in Sample States tor B-6 
Obtaining Curriculum Resources 

Uses of Computer Communications Capability B-7 
by Sample States 

Mechanisms Used by SLRs in Performing B-8 
Outreach to State and Local Users 

SLR Outreach Priorities by User Categories b-9 

SLR Reports of the Utility ot Network B-10 
Involvement 

SLR Perceptions Concerning State Use ot b-ll 
Center Resources 

SLR Perceptions Concerning Network Impacts B-12 
on Quality of State Vocational Education 
Curriculum 

SLR Reports of Technical Assistance or B-13 
Consultation Received from the Centers 

SLR Perceptions Concerning Network Impacts b-l 4 
on Reduced Duplication and Costs for 
Vocational Education Curriculum 

Areas Cited by SLRs as Needing Increased B-15 
Resources 

SLR Recommendations for Improvement ot B-16 
the Centers/Network 
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Table B-l 

SLRs* State Responsibilities (n = 15) 

State Responsibilities Number ot SLRs* 

Curriculum or program development/coordination 13 

Staff development 6 

Materials dissemination 5 

Program evaluation ? 

Program improvement/exemplary programs 6 

Other 4 



* More than one response was provided by some SLks. 
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Table B-2 



Configuration of Curriculum Resources 
in Sample States 



Number of States with Curriculum Lab or Resource Center, (n - 15) 



Lab (n - 5) Resource Center (n - 14)* 
Years in Operation 

I- 3 yrs. - 2 
4-7 yrs. 1 3 
8-11 yrs. 2 5 
12-15 yrs. - 3 
15+ yrs. 2 2 
Do not know - 1 

Staffing Level 

0-5 FTES** 1 9 

6-10 FTES 3 3 

II- 15 FTES - 1 
16-20 FTES 1 

21+ FTES - 1 



* 3 of 14 serve as both curriculum lab and resource center. 
** Full-time equivalent. 
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Sources 



Table B-3 

SLRs* Ranking of Sources Accessed in Meeting State 
Needs For Curriculum Materials (n = 15) 

Ranking* 

12 3 4 5 Total 



State lab/center 7 7 

Regional CCC 5 6 - 2 - 13 



Consortium 



2 1 1 1 5 



Another CCC - 1 1 - - 2 

NCRVE - - 1 - 1 2 

Another state/SLR 3 3 

Other • 1 1 1 3 



[ 



* Some SLRs cited fewer than 5 sources. 



78 



B-6 



Table B-4 



Mechanisms in Place in Sample States for 
Obtaining Curriculum Resources 



Flow of State and Local Requests for Information 
From Regional CCCs, (n « 14) 

Requests made directly to Centers 11 
Requests routinely channeled through SLRs 3 



Channels Used by SLRs to Obtain Information (n = 15) 

Contacts Other Regional CCCs Contacts Other SLRs 

Yes 9 13 

No 6 2 

Reasons Cited for Contacting Other CCCs, SLRS* 

CCCs (n = 9) SLRs (n = 13) 

To obtain materials 8 6 

To obtain information 6 10 

Other 2 2 



* More than one response was provided by some SLRs. 
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Table B-5 



Uses of Computer Communications Capability by Sample States 



Number of States with Computer Capacity 

for VECM/ADVOCNET , (n * 15) 

Yes 15 
No 



Mechanisms Used for Performing VECM Searches, (n = 15) 

SLR 3 
CCC 9 
NCRVE 2 
Nonresponse 1 



Reasons Cited for Requesting Searches by CCC/NCRVE, (n = ID* 



Staff not trained 3 

Takes too much time 3 

Terminal access a problem 2 

Too expensive 2 

Recently acquired capacity 2 

Other 2 



Reasons Cited for Participating in ADVOCNET (n = 15) 



Time savings/efficiency 7 

General utility 5 

Cost savings -1 

State mandated 1 

Not participating 1 



* More than one response was provided by some SLRs. 
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Table B-6 

Mechanisms Used by SLRs in Performing Outreach 
to State and Local Users 



Type of Outreach Number of SLRs* 

Local meetings/workshops 12 

State-wide conferences/meetings 7 

CCC brochures 8 

State publication 6 

Through use of state lab/center 4 

Regional communique 1 

Association meetings 1 



* More than one response was provided by some SLRs 
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Table B-7 

SLP Outreach Priorities by User Categories (n = 15) 

Priority Rankings* 
User Categories 1. jL JL i- Total 

LEA 9 4 2 - 15 

SEA 6 7 2 - 15 

Post secondary 7 5 1 1 14 

Industry 2 1 3 



* Some SLRs assigned the same priority ranking to more than 
one user category - 2 SLRs ranked first three categories as 
receiving equal priority. 
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Table B-8 

SLR Reports of the Utility of Network Involvement (n = 15) 



Nature of* 

Benefit Number of SLRs 



Professional development 14 
State job effectiveness 15 



** 

Reasons Cited for Utility of Network Involvment 



Professional State Job 

Development Ett ectiveness 
(n = 14) (n =15) 



Increased awareness 11 14 

Increased interaction 
with other vocational 

educators 7 3 

Other 3 3 



* Closed ended questions asked SLRs directly whether the 
SLR experiences had produced either of these two specif 
benefits. 

** More than one reason was provided by some SLRs. 
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SLR Perceptions Concerning State Use ot Center Resources 



Table B-9 

SLR Perceptions Concerning State Use ot Center Resoui 

Extent of State Use, (n - 15) 

Optimal state use 5 
Less than optimal state use 10 

Reasons Cited for Less Than Optimal Use (n = 10)* 

Insufficient time 5 

Insufficient awareness of Center in state 4 

Other Centers more useful 2 

Other 4 



* More than one reason was provided by some SLRs. 
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Table B-10 

SLR Perceptions Concerning Network Impacts on guality 
State Vocational Education Curriculum (n « 15) 

Nature of Improvement* 

Increased pool of available materials 

Sharing of ideas/i nf ormat ion 

Increased utilization of existing materials 

Increased state capacities through technical assistance 

Improved curriculum development generally 

Other 

Has not improved as result of Center 



* More than one improvement was cited by some SLRs. 
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Table B-ll 



SLR Reports of Technical Assistance or Consultation 
Received From the Centers (n « 15) 



Areas of TA/Consultation Received* 

High tech occupations curriculum (e.g., robotics) 2 

Competency-based curriculum 2 

General curriculum development 2 

Traditional occupational area curriculum 2 

Career exploration program 1 

Nonresponse 2 

No technical assistance/consultation received 6 



* More than one area was cited by some SLRs. 
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Table B-12 



SLR Perceptions Concerning Network Impacts on 
Reduced Duplication and Costs for 
Vocational Education Curriculum 
(n * 15) 



Impact Areas SLRs Reporting 

Reduced duplication 15 

Reduced costs 12 

Means of Achieving Reduced Costs, (n = 12)* 

Saved development costs 9 

Saved staff time 2 

Previewing of materials 2 

Generally more efficient 2 

Reasons Cited for Lack of Reduced Costs, ( n = 3) 

Would not develop anyway 1 

Can not attribute savings to Center 1 
alone 

Nonresponse 1 



* More than one response was provided by some SLRs, 
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Table B-13 

Areas Cited by SLRs as Needing Increased Resources 

Areas Needing Increased Resources (n » 15)* 

BETTER COVERAGE OF OCCUPATIONAL AREAS J 
High tech. occupations 
Marine occupations 
Hotel/motel 
Industrial arts 
Textiles 

Shoe manufacturing 
Electronics 

BETTER COVERAGE OF SPECIAL NEEDS POPULATIONS 
Handicapped 
Asian language 
Limited-English-proficient 
Single head of household 



* More than one response was provided by some SLRs. 
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Table B-14 



SLR Recommendations for Improvement of the 
Centers/Network (n * 15) 



Recommendations Number ot SLRs 

FEDERAL ADMINISTRATION 9 

Increased funding for Centers 3 

Improved continuity from one contract period to 

the next 2 

Improved consistency across Centers , functionally 1 

Reduced administrative reporting 1 

Realign regions according to state similarities 1 

Improve regional/national coordination among Centers 1 

CENTER MANAGEMENT AND OPERATIONS 7 

Increased use of computers 3 
Screen incoming materials tor quality 1 
Disseminate copies of all materials to all states 1 
Involve more people in regional meetings 1 ■ 

More concentration on library services 1 

STATE PARTICIPATION 5 

Increase number of states providing materials 

to Centers 3 

Increased support from State Directors 2 

NO RECOMMENDATIONS - SATISFIED AS IS 2 
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APPENDIX C 



TABULATIONS OF SELECTED ITEMS FROM INTERVIEWS WITH 
STATE AND LOCAL USERS 



Table C-1: 
Table C-2: 
Table C-3: 

Table C-4 : 
Table C-5: 
Table C-6: 
Table C-7: 
Table C-8: 



Pa 9 e 

State and Local User Awareness of Centers/ C-2 
Ne two rk 

State and Local Use of Center-Provided C-3 
Materials 

State and Local User Perceptions Concerning C-5 

the Utility and Quality of Materials 

Received 

State and Local User Alternatives to Center C-6 
Provided Materials 

State and Local User Receipt of Curriculum C-7 
Related Services 

State and Local User Reports of the Utility C-8 
of Curriculum Related Services 

State and Local User Alternatives for C-9 
Curriculum Related Services 

State and Local User Recommendations for C-10 
Improvement of Centers/Network 
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Table Ol 

State and Local User Awareness of Centers/Network 





Extent of User Awareness 








Local 
( n = 30) 


State 
(n ■ 29) 


T.E. 

(n = 14) 


Total 
(n = 73) 


Knows of Center 


23 


27 


14 


64 


(88%) 


Knows SLR only 


7 


2 




9 


(12%) 




Means 


of Awareness 








Local 
(n = 23) 


State 
(n = 27) 


T.E. 
(n = 14) 


Total 
(n = 64) 


Through SLR 


4 


12 


1 


17 


(27%) 


State lab/center 


1 


2 


1 


4 


(6%) 


State publications 


3 


1 


3 


7 


(11%) 


State position 


2 


9 


2 


13 


(20%> 


Presentations/meetings 


3 


1 




4 


(6%) 


Through someone other 
than SLR 


6 


1 




7 


(11%) 


Directly through Center 




1 


1 


2 


(3%) 


Do not recall 


4 




6 


10 


(16%) 


User 


Procedures for 


Contacting 


Centers 








Local 
(n ■ 30) 


State 
(n = 28) 


T.E. 

(n = 14) 


Total 
(n = 72) 


Contacts Center directly 


14 


11 


9 


34 


(47%) 


Requests channeled 
through SLR 


16 


17 


5 


38 


(53%) 
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Table C-2 

State and Local Use of Center-Provided Materials 

Receipt of Center-Provided Materials 

Local State T.E. Total 

(n = 30) (n * 29) (n = 14) (n = 73) 

Have received materials 30 27 14 71 (97%) 

Have not received materials - 2 - 2 (3%) 

Types of Materials Received * 

Local State T.E. Total 

(n » 30) (n = 27) (n = 14) (n = 71) 

General curriculum and , n ^ a .s 

instructional materials 27 . 27 14 68 (96%) 

Audio visual materials 5 1 17 (10%) 

Research/evaluation materials 2 1 - 3(4%) 



* More than one response was provided by some respondents. 
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Table 02 (continued) 



State and Local 


Use 


of Center-Provided 


Materials 








Uses of 


Materials Received* 








Local 
(n * 30) 


(n = 27) 


T.E. 
(n = 14) 


Total 
(n ■ 91) 


MATERIALS PREPARATION 




19 


14 


3 


36 


(51%) 


Curriculum development 




8 


5 


3 




16 


Reference guide 




7 


4 


- 




11 


Program development 




4 


4 






8 


Task lists/analyses 






1 


- 




1 


PT OW THROUGH 




7 


12 


4 


23 


(32%) 


Distributed to students 




" 


- 


2 




2 


Distributed to teachers 




6 


10 


2 




18 


Distributed to school 

Hi cf t*i nf 




1 


i 






2 


Distributed to state 
specialist 






1 


- 




1 


TRAINING/INSTRUCTION 




7 


6 


10 


Z3 


(32%) 


Instruction 




5 


2 


10 




17 


Staff development 




2 


2 






4 


Personal development 






2 






2 


OTHER 




1 


3 


3 


1 


(10%) 


Proposals 






1 






1 


Policy development 






2 






2 


Supplemental information 




1 








1 


Research 








3 




3 



* More than one response was provided by some respondents. 
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Table C-3 



State and Local User Perceptions Concerning the 
Utility and Quality of Materials Received 



Utility of Materials Receivec 



Very useful 
Generally useful 
Utility varies 
Can not say 



High quality 
Quality varies 
Not high quality 
Can not say 



Local 
(n = 30) 

16 

9 

4 

1 



State 
(n = 27) 

16 

7 

2 

3 



T.E. 
(n = 14) 

7 

4 

2 

1 



Quality of Materials Received 



Local 
(n = 30) 

27 

1 

1 

1 



State 
(n = 27) 

22 

3 



T.E. 
(n = 14) 

13 

1 



Total 
(n = 71) 

39 (55%) 

20 (28%) 

8 (11%) 

5 (7%) 



Total 
(n = 71) 

62 (87%) 

5 (7%) 

1 (1%) 

3 (4%) 
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Table C-4 



State and 


Local User Alternatives 
Provided Materials 


to Center- 








Alternative 


Courses 


of Action 








Local 
(n = 30) 


State 
(n • 27) 


T.E. 
I n - 1* ; 


Total 
(n = 71) 


Look elsewhere 


13 


16 


12 


46 


(65%) 


Develop 


8 


6 


1 


15 


(21%) 


Do without 


4 


4 




8 


(11%) 


Do not know 




1 


1 


2 


(3%) 


Alternative Sources of Curriculum Materials* 




Local 
(n » 30) 


' State 
(n = 27) 


T.E. 

(n = 14) 


Total • 
(n ■ 71) 


State dept/lab/c? r.ter 


7 


3 


3 


13 


(18%) 


Local school system 


2 


1 


1 


4 


(6%) 


Other states 


7 


7 


3 


17 


(24%) 


NCRVE 


5 


4 


1 


10 


(14%) 


Colleges/unive rsities 


5 


4 


3 


12 


(17%) 


Other CCCs 




1 


1 


2 


(3%) 


Private organizations 


4 


4 


2 


10 


(14%) 


Database (e.g., ERIC) 


8 


4 


1 


13 


(18%) 


Catalogs 


1 


1 




2 


(3%) 


Consortia . 


8 


4 


4 


16 


(23%) 


Other 




2 


5 


7 


(10%) 


None 


4 


2 


1 


7 


(10%) 



* More than one source was cited by some respondents. 



95 



C-7 



Table C-5 



State and Local User Receipt of Curriculum 
Related Services 



Receipt of Curriculum Related Services 

Local State T.E. Total 

(n = 30) (n - 28) (n * 14) (n * 72) 

Have received services 18 15 7 40 (56%) 

Have not received services 12 13 7 32 (44%) 

Types of Services Received * 

Local State T.E. Total 

(n = 30) (n = 29) (n = 14) (n = 73) 

Materials search 9 ' 7 3 19 (48%) 

Inservice/tech. asst./ 

consultation 11 7 3 21 (53%) 

Conference/presentation 1 3 1 5 (13%) 

Orientation/familiari- 
zation visit - - 11 (3%) 



* More than one type of service was received by some 
respondents. 
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Table C-6 

State and Local User Reports of the Utility of 
Curriculum Related Services 



Utility of Curriculum Related Services * 

Local State T.E. Total 

(n = 18) (n = 15) (n = 7) (n_=4p) 



Savings of time/ef f ort/ 

money 10 12 4 26 (65%) 

Improved curriculum 

development 5 2 18 (20%) 

Increased awareness 7 6 13 (33%) 

Staff development 6 2 19 (23%) 

Facilitates planning 2 1 3 (8%) 

Other - - 3 3 (8%) 

Was not useful 1 - - 1(3%) 



* More than one response was provided by some respondents. 
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Table C-7 



State and Local User Alternatives for Curriculum 

Related Services 



Alternative Sources for Curriculum Related Services* 



State department 
Other states 
Consortia 

NCRVE 

Colleges/universities 
Database (e.g., ERIC) 
Private organization 
Trade/prof, association 
Other 

No other source 
available** 



Local 
(n = 18) 



3 
2 
2 



State 
(n = 15) 

1 

3 

1 

4 

3 
1 

1 
3 



T.E. 

(n = 7) 



3 
2 
2 



Total 
(n = 40) 

5 (13%) 



3 
2 
4 



(8%) 
(5%) 
(1%) 

6 (15%) 

7 (18%) 
5 (13%) 
1 (1%) 
3 (8%) 



11 (28%) 



* More than one source was cited by some respondents. 

** Technical assistance or presentations are services refer- 
enced by 8 of the 11 respondents citing no alternative sources. 

L 
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Table C-8 

State and Local User Recommendations for Improvement 

of Centers/Network 



Recommendations 

ORGANIZATIONAL/STRUCTURAL 

Realign regions by state 
similarities 

Mors staff at Centers 

Clearer definition of 
Center role 

Move Center to another 
s tate 

STATE PARTICIPATION 

Increase state outreach 

Increase state materials 
sharing 

CONTENT OF MATERIALS 

More high tech 

More adult education 

More Hispanic/LEP 

More leadership 

More current materials 
FORMAT OF MATERIALS 

More sophisticated 

More CBVE emphasis 

Briefer materials 

More slides 



Local 
(n = 30) 



12 
12 



2 
1 



1 
1 



State 
(n =29) 



1 
5 
4 

1 
3 



3 
2 
1 
1 



T.E. 
(n = 14) 



1 
4 
4 



Total 
(n = 73) 

7 (10%) 



2 
1 



21 (29%) 
20 



8 (1 



'%) 
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Table C-8 (continued) 

State and Local User Recommendations for Improvement 

of Centers/Network 



Recommendations 

AVAILABILITY OF MATERIALS 
More materials generally 
Increase use of catalogs 



Local 
(n = 30) 

8 

1 

1 



Have all materials 

accessible via computer 

Easier access to materials 3 

More V-TECS materials 1 

More copies of materials 2 

SERVICES 4 

More workshops/i nse rvice 1 

Regional meetings of user? 1 

Increase use of electronic 

ma i 1 1 

Coordinate public/private 

curriculum development 1 

NO RECOMMENDATIONS 8 



State 
<n = 29) 



3 
2 



1 
9 



T.E. 
(n =14) 



Total 
(n = 73) 

14 (19%) 

1 

3 



6 
1 
2 

8 (11%) 
4 

1 



22 (30%) 
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APPENDIX D 
IMPACT REPORTING 

Center Directors are required to provide OVAE with periodic 
progress reports and annual summary impact reports. Key items 
from the six Centers' impact reports are aggregated by OVAE tor 
use in the annual Network -wide Curriculum Coordination Center 
Impact Report. The content of the Centers' impact reports, 
specifically the standard data items that are used and the 
standardized formulas for estimating the value ot certain 
activities, are prescribed by the Directors' Council. 

Impact reporting currently is based on 32 data items or "key 
results areas" organized under three functional headings: (1) 
coordination and management, (2) curriculum services, and (3) 
dissemination and utilization. SLRs voluntarily report to their 
Centers on the subset of items that pertain to state- level 
act ivi ty , and Center Directors compile these reports and 
incorporate data reflecting Center- level activity . 

One of the tasks of our study was reana lysis ot Center 
reported impact data as one source of evaluative information. In 
performing this task we encountered several obstacles, including 
the following : 

• Not all Centers report information to OVAE on a 
state-by-state basis. For this reason, the impact 
data could not be used to interpret the effects 
individual states have on overall regional activity 
or to corroborate the perceptual data obtained 
during interviews. While this factor does not 
necessarily have implications for routine program 
monitoring, it did complicate attempts to verify 
information from the various data sources used in 
the study. 
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+ Measures of Center activities are (or at least 
appear to be) merged with measures of SLR activities 
for selected items. Thus it is not possible to use 
the data to assess the effectiveness of Center-level 
management of federal funds. The data are further 
confounded by the variation in job responsibilities 
of individual SLRs. An example of this problem is 
found in the measures "Inservice workshops and 
attendees," which aggregate workshops conducted or 
involving either Center staff or SLRs. For purposes 
of understanding the federal program and the role of 
SLRs, it would have been useful to have these 
reported separately. 

e Terms appear to be interpreted differently across 
states and regions. This lack of consistency brings 
the validity of the data into question. Users ot 
the data are uncertain as to what is intended to be 
measured and whether what is being measured is what 
was intended to be measured. An example of this 
problem is the data item "Persons reached through 
dissemination, " where dissemination could be 
variably interpreted as providing materials or 
describing the Network at a conference. 

• Items combine noncomparable types ot activities. 
Thus it is difficult to draw inferences from the 
data iteir. as to what was accomplished . Examples ot 
this problem include "Items cisseminated , " which can 
include outreach materials as well as actual 
curriculum materials and "Dollars saved and cost 
benefits of the Network" which collapses dollars not 
expended as a result of adoption/adaptation with 
costs incurred through participation in Network 
act i vit ies. 

Reporting in several of the key result areas is affected by more 
than one of these discrepancies. 

The impact reporting system has evolved over time , with 
additional measures added as the Network became involved in new 
activities and new issues came to be of interest. This pattern 
is indicative of a flexible management system and one that is not 
stagnant. However, it appears that adjustments to the system 
have primarily been in the -direction ot additions and that key 
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results areas have not been deleted as they have become less 
relevant and the measures may have lost their significance* 

Prime examples of measures having little significance in 
terms of conveying annual Network impacts are those pertaining to 
the presence of state systems for vocational education curriculum 
(e.g . , library loan systems , dissemination systems , systematic 
review of materials) and those related to various SLR activities 
(e.g., participation in Regional Meetings, "systematic communi- 
cation" with other state staff). For several years now each of 
the Centers has reported the maximum or near maximum number pos- 
sible for these types of measures. State systems are established 
and over the course of the Network's existence, the SLR role has 
been defined and solidified. While certainly evidence of the 
Network's historic impact is important, those "result areas" 
relating to the presence of certain state systems no longer 
contribute to a picture of anr:ual impacts or improvement ♦ 
Similarly, measures of SLR functions or activities, wnile impor- 
tant management concerns, do not provide any insight from an 
impact or outcome perspective. 

The impact reporting system as it currently exists has some 
attributes of a management system and some of an evaluation 
system. While we were unaDle to document the history ot each key 
results area measure, we suspect that the definitional problems 
we encountered may stem from the system's attempting to accom- 
plish both objectives simultaneously. The item "consultation and 
technical assistance requests" provides a good example ot how we 
arrived at this hypothesis. 
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In the context of this evaluation we were interested in 
demand among users for various Center services (e.g., consulta- 
tion regarding how to obtain reference materials and the number 
of users perceiving the Centers to be a source ot technical 
assistance) in order to assess the effectiveness of outreach 
activities. In this regard, the combination ot concepts (TA and 
consultation) in one item limits its utility. However, from an 
internal management perspective the item may indeed be useful. 
These requests are received by phone cr letter and must be 
answered. Thus, the item allows Center Directors to monitor ana 
document changes in staffing needs over time. 

It may be time for a reassessment of the impact reporting 
system that begins with a detailed statement or its purpose, 
criticjes each measure against tnis purpose, eliminates those 
that dc not support a giver, purpose, a no re i ns only tr.cs<= that 
are central. As it is currently structured the impact reporcin^ 
systerr has the appearance of attempting to accommodate every 
activity of tne Network rather than focusing on sound reporting 
of a more limited set of highly descriptive measures. 
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